Augmented reality task flow optimization systems

ABSTRACT

A method, system, apparatus, and/or device for adjusting a task or a task flow in an augmented reality environment. The method, system, apparatus, and/or device may include: identifying a first task in a task flow to collect data associated with a user performing the first task; collecting a first set of data associated with the user performing the first task; comparing an amount of time the user expends to perform the first task to a first threshold amount of time; and in response to the amount of time the user expends to perform the first task exceeding the first threshold amount of time displaying a notification to the user or another device.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. Non-Provisional applicationSer. No. 16/422,013, filed May 24, 2019, which claims the benefit ofU.S. Provisional Application No. 62/678,211, filed on May 30, 2018, theentire contents of which are hereby incorporated by reference for allpurposes.

BACKGROUND

An increasing number of workers in a variety of industries are assistedby hand-held or wearable computer systems. Rather than using computerkiosks or workstations at locations throughout the work environment, thehand-held or wearable computers allow the worker to move freely aboutthe workspace and retrieve information from computer networks accessibleat their fingertips, while they maintain in proximity to the object ofthe task. An augmented reality system may be used to further enhancetask flows where users need information to complete a task.

BRIEF DESCRIPTION OF THE DRAWINGS

The present description will be understood more fully from the detaileddescription given below and from the accompanying drawings of variousembodiments of the present embodiment, which is not to be taken to limitthe present embodiment to the specific embodiments but are forexplanation and understanding.

FIG. 1A illustrates a perspective view of a wearable device being wornby a user, according to an embodiment.

FIG. 1B illustrates a perspective view of the wearable device in FIG.1A, according to an embodiment.

FIG. 2 illustrates an augmented reality system with a user wearing thewearable device, according to an embodiment.

FIG. 3 illustrates a system of the wearable device in FIGS. 1A-2 ,according to an embodiment.

FIG. 4 illustrates a functional block diagram for the optimizer shown inFIG. 3 , according to an embodiment.

FIG. 5 illustrates the wearable device in FIGS. 1A-4 displaying amessage associated with a task of a task flow for a project, accordingto an embodiment.

FIG. 6A illustrates a flowchart of a method for revising a task flow fora set of tasks, according to an embodiment.

FIG. 6B illustrates a flowchart for a method to adjust task in a taskflow, according to an embodiment.

FIG. 7 is a block diagram of a user device with a processing device inwhich implementations of methods, systems, devices, or apparatuses inFIGS. 1-6B may be implemented.

DETAILED DESCRIPTION

The disclosed augmented reality task flow optimization systems willbecome better understood through a review of the following detaileddescription in conjunction with the figures. The detailed descriptionand figures provide merely examples of the various embodiments describedherein. Those skilled in the art will understand that the disclosedexamples may be varied, modified, and altered and not depart from thescope of the embodiments described herein. Many variations arecontemplated for different applications and design considerations;however, for the sake of brevity, the contemplated variations may not beindividually described in the following detailed description.

Throughout the following detailed description, examples of variousaugmented reality task flow optimization systems are provided. Relatedfeatures in the examples may be identical, similar, or dissimilar indifferent examples. For the sake of brevity, related features will notbe redundantly explained in multiple examples. Instead, the use ofrelated feature names will cue the reader that the feature with arelated feature name may be similar to the related feature in an exampleexplained previously. Features specific to a given example will bedescribed in that particular example. The reader is to understand that agiven feature need not be the same or similar to the specific portrayalof a related feature in any given figure or example.

Individuals that work in industries where their hands must remain freeto perform tasks may use wearable computer systems to retrieve and/orview information needed to perform the tasks. For example, individualsworking in manufacturing facilities may be required to maintain a cleanand hygienic manufacturing area to prevent cross-contamination.Additionally, in the manufacturing environment and other environments,it may be required to record that all the steps and instructions in adefined manufacture procedure were performed. Wearable computer systemsmay aid in recording all the steps and instructions performed and ininvestigating or documenting deviations taken during the manufacturingprocedure. The records of the manufacturing and distribution of itemswith a complete manufacturing history may enable a manufacturer toretain the manufacturing information in a comprehensible and accessibleformat in case an investigation is needed to determine a product defector failure. In another example, in retail operation environments, salesassistants or inventory control clerks may use wearable computer systemsto identify products and/or retrieve product information associated witha product. The wearable computer system may provide guidance and aidindividuals in performing tasks. However, different users may performdifferent tasks or the different users may perform the same taskdifferently. Providing the same guidance and aid to the different usersmay be ineffective for the individuals who perform the task differentlyfrom the default guidelines. Additionally, providing the same guidanceand aid to the different users may not aid the individuals in learninghow to correctly perform tasks.

Implementations of the disclosure address the above-mentioneddeficiencies and other deficiencies by providing methods, systems,devices, or apparatus to display an augmented reality environment tousers and provide interactive user interfaces. The methods, systems,devices, or apparatus may also improve a task flow of tasks displayed inan augmented reality system.

In one embodiment, the augmented reality systems may be head-mountedsystems. In another embodiment, the augmented reality systems mayinclude sensors and/or detector units to detect, gather, and/or transmitdata to a processing device. The processing device may be an integratedor onboard processing device or an external or remote processing device.In another embodiment, the augmented reality systems may include anaugmented reality display to display or overlay information to a user.In another embodiment, the augmented reality systems may includeproviding a user access to remote help, capture images and/or sensordata to send to a remote individual, and so forth. For example, anaugmented reality system may transmit a still image or a video from aperspective of the user, also referred to as a “see what I see”perspective.

In another embodiment, the augmented reality systems may overlay imagesfrom another user's device into an augmented reality environment displayfor a viewer to see a projected perspective of the images from the otheruser's device, such as a re-creation of what is being viewed by theother user. One advantage of the augmented reality systems may be toenhance task flows of users when the users need information displayed tothem to perform a task. Another advantage of the augmented realitysystems may be to provide a user interface for the users to interactwithin a variety of environments, include hands-free environments, cleanand hygienic environments, collaborative environments, remoteenvironments, and so forth.

FIG. 1A illustrates a perspective view of a wearable device 100 beingworn by a user 102, according to an embodiment. The wearable device 100is designed to display an image near an eye or eyes of a user 102 andexecute computer-executable instructions in response user gestures byincorporating motion and image capture features. The wearable device 100may include lens 104 substantially near the eye(s) of the user 102. Thelens 104 may include a transparent or partially transparent screen thatis at least partially disposed within a field of view of the user 102.The screen may display an augmented reality where a user may be able toview augmented reality objects. The screen may have a transparency levelto enable the user 102 to see gestures 106 and real-world objects 108.

FIG. 1B illustrates a perspective view of the wearable device 100 inFIG. 1A, according to an embodiment. Some of the features in FIG. 1B arethe same or similar to some of the features in FIG. 1A as noted by samereference numbers, unless expressly described otherwise.

The wearable device 100 may include a first temple 110, a second temple112, and a bar 114 extending between the two temples 110 and 112. Thefirst temple 110 and second temple 112 extend from the wearable device100 to rest on the ears of the user 102 in FIG. 1A to provide supportfor the wearable device 100.

The wearable device 100 may include a processing device 116, the lens104, a first image generator 118, a second image generator 120, a motioncapture device 124, and a microphone 122. The wearable device 100 may beconfigured to capture user gestures, along with other objects within acaptured scene, and execute computer processes or commands in responseto the captured objects. This may involve manipulating a displayproduced by a first image generator 118 and/or a second image generator120 on lens 104. For example, the first image generator 118 may displaya first virtual object or portion of a virtual object on a first portionof the lens 104. In another example, the second image generator 120 maydisplay a second virtual object or portion of a virtual object on asecond portion of the lens 104. In another example, the lens 104 may bea single continuous lens extending across the bar 114. In anotherexample, the lens 104 may include multiple lens or portions of lensextending across the bar 114. The first virtual object and/or the secondvirtual object includes text, a graphical object, a video, a live datasteam, and so forth.

The wearable device 100 may additionally execute computer processes inresponse to audio captured by the microphone 122. By incorporating thesecapturing technologies, the wearable device 100 may display and controlcomputer images and processes. The processing device 116 may besupported by the bar 114. The processing device 116 may be configured toexecute computer-executable instructions and control the electronicelements of the wearable device 100.

The processing device 116 may acquire data from other elements of thewearable device 100 or from external sources and execute computerexecutable code in response to this data. For example, the processingdevice 116 may be configured to acquire data from the motion capturedevice 124 such as data that corresponds to a user gesture. Theprocessing device 116 may additionally or alternatively acquire datafrom the microphone 122. In some examples, the processing device 116 mayacquire data from a separate device, such as a portable music player, apersonal data assistant (PDA), a smartphone, a global positioning system(GPS), or the like.

The processing device 116 may be in coupled to the first image generator118 and/or the second image generator 120 and may instruct the firstimage generator 118 and/or the second image generator 120 to generateand manipulate a display projected on lens 104. In one example, theprocessing device 116 may generate a user interface on the lens 104. Theprocessing device 116 may acquire data from other elements or sensors ofthe wearable device 100 and manipulate the user interface in response tothis data.

The first image generator 118 and/or the second image generator 120 maybe attached to the first temple 110 and may be substantially alignedwith the lens 104. The wearable device 100 may include additional imagegenerators to augment the expanse of generated images over the lens 104surface area. The first image generator 118 and/or the second imagegenerator 120 may be configured to display images on the lens 104 facingan eye or eyes of the user 102 in FIG. 1A in response to instructionsexecuted by the processing device 116. These images often may overlayand/or augment a naturally viewed scene within a field of vision of theuser 102. The first image generator 118 and/or the second imagegenerator 120 may display opaque and/or partially transparent images onthe lens 104.

The motion capture device 124 may be connected to the bar 114. In oneembodiment, the motion capture device 124 may be oriented away from theuser 102 when the user 102 is wearing the wearable device 100 to captureimages and motion occurring beyond the lens 104. In another embodiment,the motion capture device 124 may include an image sensor 126 with acamera, a light source 128, and a depth sensor 130. The motion capturedevice 124 may capture images that include at least a portion of theenvironment surrounding the wearable device 100 (such as a head-mounteddevice or a head-mounted display).

The image sensor 126 may be connected to the bar 114. In one embodiment,the image sensor 126 may be oriented away from the user 102 as the user102 wears the wearable device 100. The image sensor 126 may beconfigured to capture an image from a scene and communicate datacorresponding to the captured image to the processing device 116. Theimage sensor 126 may capture light within the visual spectrum and/orlight outside the visual spectrum. For example, the image sensor 126 maycapture infrared or ultraviolet light.

The light source 128 may be connected to the bar 114. The light source128 may be substantially aligned with the image sensor 126. The lightsource 128 may be configured to project light in a defined direction.This light is reflected from an object and may be captured by the motioncapture device 124. The reflected light may allow the motion capturedevice 124 to more accurately capture images and motion of objects orthe surrounding environment. The light source 128 may project visuallight and/or light outside the visual spectrum.

The depth sensor 130 may be connected to the bar 114. The depth sensor130 may be configured to capture images and motion from a scene at adifferent angle than image sensor 126. The data captured from thissecond viewpoint may allow the processing device 116 to compare the datareceived from the depth sensor 130 to the data received from the imagesensor 126 to better detect a depth of objects in the environmentsurrounding the wearable device 100.

FIG. 2 illustrates an augmented reality system 200 with a user 102wearing the wearable device 100, according to an embodiment. Some of thefeatures in FIG. 2 are the same or similar to some of the features inFIGS. 1A and 1B as noted by same reference numbers, unless expresslydescribed otherwise. The user may be located along a first plane 204.The wearable device 100 may include a lens 104 to display a virtualobject to an eye or eyes of the user 102. For example, the wearabledevice 100 may include a first portion of the lens 104 that may displaya first image or virtual object to a first eye of the user 102 and asecond portion of the lens 104 that may display a second image orvirtual object to a second eye of the user 102. When combined oraggregated, the first image or virtual object and the second image orvirtual object may form an augmented reality environment to project oneor more virtual or augmented reality objects to the user 102 at definedlocations within a physical environment surrounding the user 102.

The wearable device 100 may display multiple images to project one ormore virtual or augmented reality objects 208 within a physicalenvironment surrounding the user 102. In one embodiment, the user andthe wearable device 100 may be located along the first plane 204 and theaugmented reality object 208 may be displayed to appear at a defineddistance from the user 102 at a second plane 206. In one embodiment, thewearable device 100 may include a position sensor, such as a GPS deviceor a position transceiver. For example, the wearable device 100 may be apair of glasses, smart glasses, a face shield, and so forth that may beworn by the user 102 and include multiple sensors, including theposition sensor. In another embodiment, the position sensor may separatefrom the wearable device 100 and be attached to the body of the user102.

In one example, the position sensor may be positioned proximate to aline of sight of the user 102. In another embodiment, an alignmentsensor may measure a difference between a position of the user 102indicated by the position sensor and the actual location of the user'seyes and generates a view aligned to the user's line of sight. In oneembodiment, the lens 104 in FIGS. 1A and 1B may project a virtual oraugmented reality environment or a virtual or augmented reality object208 at a set focal distance from the user 102. In one example, the focaldistance may be along the second plane 206. The virtual or augmentedreality environment or the virtual or augmented reality object 208 maybe a user interface that enhances or supplements the user's interactionsin the physical environment and/or physical objects approximate orsurrounding the user.

FIG. 3 illustrates a system 300 of the wearable device 100 in FIGS. 1A-2, according to an embodiment. The system 300 may include a detector 302,sensors 304, a transceiver 306, an augmented reality display 308, alocal processing device 310, a remote processing device 312, and anoptimizer 314. In one embodiment, the detector 302 and/or the sensors304 may take measurements and collect data from a physical environmentapproximate to the wearable device 100. In another embodiment, the localprocessing device 310 may be a local processing device attached to orintegrated into the wearable device 100.

The local processing device 310 may be positioned proximate to the userthat interacts with a remote processing device 312. In anotherembodiment, the remote processing device 312 may be a remote processingdevice that may be external to the wearable device 100. The transceiver306 may receive instructions from the local processing device 310 toproject the augmented display 20. The transceiver 306 may measurecharacteristics from the physical environment and communicate the datato the local processing device 310 and an optimizer 314.

The optimizer 314 may be in communication with the remote processingdevice 312, the local processing device 310, and the transceiver 306.The transceiver 306 may measure or detect user characteristics orgestures including head position, movement, speed, etc. In one example,using gestures, the user may open applications to be displayed in theaugmented environment. These applications may include task flows,attached documents, and so forth. The gestures may allow the user toaccess the applications and execute instructions such as selectinstructions, open instructions, scrolling instructions, movementinstructions, and so forth. Alternatively, the gestures may be used tooperate on the icon representing the application by changing the size(zooming in and out), color, sound, and so forth. The gestures mayinclude the user performing scrolling gestures, tapping gestures, orclicking gestures with respect to a virtual object. The scrollinggestures may include body motions, head motions, hand motions, and soforth.

FIG. 4 illustrates a functional block diagram for the optimizer 314shown in FIG. 3 , according to an embodiment. In one example, thedatabase 412 may include unique user profiles, industry standards, andso forth. The optimizer 314 may include a memory device to store userprofile module 402. In one example, the user profile information mayinclude information about a user's age, job or responsibilities,experience level, gender, length of employment, previous tasksperformed, and so forth. The optimizer 314 may include a time thresholdindicator 404 with a database to store information indicating athreshold amount of time an individual should take to complete a giventask when the task is performed efficiently and correctly. For example,when the user is working on changing the oil of an airplane, thedatabase of the time threshold indicator 404 may include defined amountsof time for the tasks of removing an oil plug, draining the oil,removing the oil filter, inserting a new oil filter, replacing the plug,and filling the oil pan with oil. In one embodiment, the thresholdamounts of time may be set by an expert or an individual experienced inperforming the tasks. In another embodiment, the threshold amounts oftime may be determined by averaging an amount of time taken by a set ofusers each performing the tasks, such as during a training period forthe optimizer 314. The threshold amounts of time may be updated orchanged over time as the average changes or, as discussed below, basedon user profile information.

The optimizer 314 may include a comparator 406 that is configured toreceive the user profile information from the user profile module 402,the threshold amounts of time for the individual to perform the tasksfrom the time threshold indicator 404, and the actual amounts of timetaken by the user to perform the tasks as measured by a sensor 414 ofthe optimizer 314. The sensors 414 may be the sensors of the wearabledevice 100 as in FIGS. 1A-3 . The comparator 406 may determine whetherthe actual amount of time taken by the user to perform the tasks iswithin the threshold amounts of time.

In one embodiment, the threshold amounts of time may be fixed, where thethreshold amounts of time are the same for any individual performing thetasks. For example, the threshold amount of time for removing the oilplug may be 3 minutes for any individual performing the task and thecomparator 406 may determine whether the individual performed the taskwithin the 3 minutes of allotted time. In another example, thecomparator 406 may determine if the individual exceeds the 3-minutethreshold amount of time and if so by how much. In another example, thecomparator 406 may determine if the individual performed the task inless then the 3 minutes of allotted time.

In one example, the threshold amounts of time may be adjusted based onthe user profile. For example, when the user profile indicates that theindividual is relatively inexperienced for performing the task, thecomparator 406 may use a first threshold amount of time instead of asecond threshold amount of time associated with an individual that isrelatively experienced or has a user profile with an experience levelthat exceeds a threshold level. In another embodiment, the comparator406 may increase or decrease the threshold amount of time-based oncharacteristics of the user indicated in the user profile, such asexperience level, age, gender, and so forth. For example, when theindividual is an inexperienced elderly individual the comparator 406 mayincrease the threshold amount of time whereas when the individual is anexperienced youthful individual the comparator 406 may decrease thethreshold amount of time.

In another embodiment, the sensor 414 or another sensor may takeenvironmental measurements, such as light conditions measurements,weather conditions measurements, temperature conditions measurements,time of day indicators, noise levels measurements, and so forth. Thecomparator 406 may adjust the threshold amount of time-based on theenvironmental measurements. For example, when environmental measurementsindicate that the light level is low, the noise level is high, or theweather is raining or snowing, the comparator 406 may increase thethreshold amount of time for the individual to complete the task. Inanother example, when environmental measurements indicate that the lightlevel is normal, the noise level is low, or the weather is sunny, thecomparator 406 may decrease the threshold amount of time for theindividual to complete the task.

In another example, the comparator 406 may store the informationregarding whether the individual performed the tasks within thethreshold amount of time and/or the difference between the time theindividual performed the tasks and the threshold amount of time at thedatabase 412. In one example, the user profile module 402 may be coupledto the database 412. The user profile module 402 may access the database412 to retrieve the information indicating the amount of time theindividual took for a task or a set of tasks in comparison to thethreshold amounts of time for the task(s), and the user profile module402 may update the user profile for that individual to reflect thatinformation.

The optimizer 314 may include a suggesting module 408 that is coupled tothe user profile module 402. In one example, the suggesting module 408may access the user profile information stored at the user profilemodule 402. A module may be an application or software executing on aprocessing device. In another example, the suggesting module 408 mayaccess the user profile information at the database 412 via the userprofile module 402. In another example, the suggesting module 408 mayaccess the database 412 directly. In one embodiment, when the userprofile information indicates that the time the user is taking tocomplete a task exceeds the threshold amount of time, the suggestingmodule 408 may provide a hint or recommendation on how to complete thetask.

In another embodiment, when the user profile information indicates thatthe time the user is taking to complete a task exceeds the thresholdamount of time, the suggesting module 408 may provide the user with atutorial video or image to aid the user in completing the task. Inanother embodiment, when the user profile information indicates that thetime the user is taking to complete a task exceeds the threshold amountof time, the suggesting module 408 may identify an expert or anindividual experienced in performing the task in a database of thesuggesting module 408 or connected to the suggesting module 408. Thesuggesting module 408 may then establish a communication link betweenthe wearable device 100 and a device associated with the expert or theexperienced individual so that the expert or experienced individual mayprovide the user with aid and/or recommendation in completing the task.For example, the communications link may be a video chat, an audio chat,a phone call, a text message, an email, and so forth.

In one embodiment, the suggesting module 408 or the wearable device 100may record the communication between the user and the expert in thedatabase 412, another database, or a memory storage device. The storedcommunications may include instant messages, e-mails, phone calls,files, video chats, transcripts, and so forth between the users of theuser devices.

In one example, when the amount of time the user or another user takesto perform the same task exceeds the threshold amount of time for thetask, the suggesting module 408 may playback, to the user or the otheruser, the recorded communication between the user and the expert. Inanother example, the suggesting module 408 may update a task flow forthe user to perform the task based on the communications between theuser and the expert and/or other aids or recommendations received by theuser. In another example, the suggesting module 408 may monitor how theuser adjust or changes how he/she performs the task based on thesuggestions or recommendations and revised the task flow in view of theadjustments or changes.

As further discussed below, the suggesting module 408 may sendnotifications to a system administrator or the expert that manages thethreshold amounts of time. The notifications may include recommendationsor suggestions to increase or decrease the threshold amounts of time fordifferent tasks based on the amount of time taken by the user ormultiple users to perform a task. For example, if the average time a setof users take to perform a task exceeds the threshold amount of time,the suggesting module 408 may send a suggestion of the systemadministrator or the expert to increase the threshold amount of time.

In one embodiment, the suggesting module 408 may be connected to aremote processing device 410. The suggesting module 408 may send arecord of the notifications, sensor data, and/or other informationcollected and/or recorded by the optimizer 314 to the remote processingdevice 410. In one embodiment, the remote processing device 410 mayassociated with a task flow owner so that the task flow owner may reviewthe notifications, the sensor data, or the other information to improvethe task flow. In another embodiment, the remote processing device 410may be coupled to the use profile module 402 and the remote processingdevice 410 relay the notifications, the sensor data, or the otherinformation to the user profile module 402. The user profile module 402may then update the user profile associated with the user based on thenotifications, the sensor data, or the other information. In anotherembodiment, the suggesting module 408 may be coupled to the user profilemodule 402 and may send the notifications, the sensor data, or the otherinformation directly to the user profile module 402.

FIG. 5 illustrates the wearable device 100 in FIGS. 1A-4 displaying amessage associated with a task of a task flow for a project, accordingto an embodiment. Some of the features in FIG. 5 are the same or similarto some of the features in FIG. 1A-1B as noted by same referencenumbers, unless expressly described otherwise. As discussed above, theoptimizer 314 in FIGS. 3 and 4 may collect and analyze a task(s) of atask flow for an individual performing one or more tasks and providerecommendations to improve the task and/or task flow. For example, themethod may provide steps for optimizing a task flow of the user in theaugmented reality environment. The task flow may be a task-by-taskprocess showing how a user may interact with a system in order tocomplete a project. As discussed below, a task flow may include adiagram or map of one or more sequences of tasks for the user tocomplete the project and decision points for the user to followdifferent sets of tasks for the user to follow to complete the projectbased on the activities and decisions of the user.

The tasks and decision points may represent activities performed by theuser and/or a system in completing the project. The wearable device 100may display a notification or message 504 to a user to instruct the userregarding the next task in the task flow and/or display notifications toaid the user in completing the task within a threshold amount of time.For example, when the user is struggling to complete a task and hasexpended an amount of time that exceeds a threshold amount of time, thedisplay may display the notification 504 to provide the user with asuggestion on how to complete a task for object 502. The notification504 may include text, graphical objects, videos, video chats, live datastreams, and so forth. The notification 504 may aid the user to completethe task and/or provide feedback to the user after the user completesthe task.

FIG. 6A illustrates a flowchart 600 of a method for revising a task flowfor a set of tasks, according to an embodiment. In one embodiment, themethod in the flowchart 600 may be performed by the optimizer 314 inFIGS. 3 and 4 executing on a processing device. The method may includeidentifying a task in a task flow to collect data associated with a userperforming the task (block 602).

In one embodiment, to identify the task to collect data for, aprocessing device may identify the task based on one or more parametersor criteria. In one example, the parameters may include an experiencelevel or skill level of the user, industry-specific restrictions, and soforth. For example, when the individual is a trainee with aninsufficient amount of skills or experience to perform the task, theprocessing device may determine not to collect data associated with theuser performing the task. In another example, when the individual isrelatively experienced, the processing device may identify the task tocollect data on. In another example, when the task includes informationassociated with privacy rules, such as patient privacy, user privacy, ortrade secrets, the processing device may not collect data associatedwith the task or may only collect data for certain tasks.

In another embodiment, the collected data may include performanceinformation, speed measurements, a number of movements by theindividual, a size of motion or movements by the individual (measured bydistance or angle measurements), environment information, and so forth.In another embodiment, the type of data collected for the task may beadjusted based on the individual performing the task. For example, thewearable device 100 in FIGS. 1A-5 may include one or more sensors (suchas an optical camera or infrared scanner) to measure the physicalfeatures of an individual, such as a height of the individual, visualcapabilities of the individual, body type of the individual, gender ofthe individual, and so forth. When the wearable device 100 measures thephysical features, the processing device may select an initial userprofile based on the physical feature of the individual. In anotherexample, based on the physical features of the user, the processingdevice may suggest physical aids for the user to utilize via a display,such as using stepstool or repositioning the machinery. In anotherembodiment, the task may be identified by a system administrator or userfor optimization of the task or for further review by the systemadministrator. In another embodiment, the system administrator or usermay identify tasks that were previously optimized or reviewed that donot need further review or optimization. In another embodiment, theselected tasks may be chosen by a system administrator or user based ona priority of the task being performed by the user.

The method may include displaying a message associated with the task(block 604). For example, the task flow may be a task-by-task process ofactivities a user is to perform to complete a project. In one example, atask flow may include a diagram or map of the one or more sequences oftasks for the user to complete the project and decision points for theuser to follow different sets of tasks for the user to follow tocomplete the project based on the activities and decisions of the user.The tasks and decision points may represent activities performed by theuser and/or a system in completing the project. In one embodiment, thewearable device 100 of FIGS. 1A-5 may include a display with a userinterface to indicate one or more of the tasks the user is to performand/or activities associated with the one or more tasks to complete thetasks in the task flow for the project.

The method may include identify a data model based on a user profile ofthe user (block 606). The data model may be a model of an execution timeof the task by an expert or an average of all task performers. In oneembodiment, an initial data model may be defined by an expert inperforming the task. In one example, when the initial data model isdefined, the processing device may update the model based on usersperforming the task. The processing device may adjust the data modelbased on an average of time taken for users to perform the task. Thedata model may be iteratively updated as the number of users thatperform the task increases and/or the number of times the users performsthe task increases.

The method may include comparing an amount of time the user expends toperform the task to a threshold amount of time associated with the datamodel (block 608). For example, an expert may define an optimal amountof time for a task as being 5 min to execute. In one embodiment, whenthe user finishes the task in two minutes, the processing device may tagthe data as indicating a user completed the task in less than thethreshold amount of time. In another embodiment, when a threshold numberof users complete the task in an amount of time less than the thresholdamount of time, the processing device may decrease the threshold amountof time, such as to an average of the amount of time taken by the users.For example, when a first user completes the task in 2 minutes and asecond user completes the task in 3 minutes, the processing device mayset the threshold amount of time as 2.5 minutes. In another example, theprocessing device may select the highest amount of time taken by a userthat is less than 5 minutes and may set the threshold amount of time asthat amount of time.

In another embodiment, when the amount of time taken by the user isapproaching the first threshold amount of time (i.e. within a secondthreshold amount of time) the processing device may provide anotification to the user suggesting the user contact an expert. When thefirst amount of time is reached, the processing device may require theuser to contact the expert or provide a second recommendation to contactthe expert. For example, for a first threshold amount of time of 5minutes, when the user has taken 4 minutes to work on the task, theprocessing device may suggest the user contact the expert. If the userignores the 4-minute recommendation and the user reaches the 5-minutethreshold, the processing device may provide a second recommendation ormay require the user contact the expert. The expert may be an individualhave experienced with the task or the overall project. For example, theindividual that originally generated the task or the task flow for theproject may be defined as an expert for the task.

When the user completing the task within the threshold amount of time,the processing device may identify another task of the task flow tomonitor. For example, the processing device may iteratively monitordifferent tasks in the task flow to increase the user's efficiency anddecrease the amount of time the user expends when performing thedifferent tasks in the task flow.

The method may include, when the user does not complete task within thethreshold amount of time, providing a first notification to the userwhile the user is performing the task (block 610). For example, when theamount of time the user takes to complete the task exceeds the thresholdamount of time, the processing device via a display may display anotification providing a suggestion for how the user may reduce the timetaken to complete the task or how to otherwise improve the user'sefficiency in performing the task. As discussed above, the thresholdamounts of time may be based on a profile of the user and/orenvironmental factors in the area where the user is performing the task.For example, when the user is performing the task on an oil rig and theambient temperature increase beyond a threshold degree level, theprocessing device may increase the amount of time the user may take toperform the task. In another example, the environmental factors mayinclude light parameters, such as a hue level, an intensity level, anillumination level, and so forth.

The method may include providing a second notification to the user whenthe task or the project is completed (block 612). In one embodiment,when the processing device via a display displayed a notification to theuser as the user performed the task, the processing device via thedisplay may display a second notification when the task or the projectis complete, where the second notification requests feedback from theuser as to whether the first notification was helpful to the user incompleting the task. For example, to determine that the firstnotification was helpful to the user, the processing device via thedisplay may determine when the user proceeds to the next step of theproject and display a notification requesting input as to whether thefirst notification was helpful. The user may then use an input device(such as a gesture detection device for an augmented realityenvironment) to input whether the first notification was helpful. In oneembodiment, the user response to the second notification may be sent toa device associated with the system administrator or another device forthe system administrator to use in adjusting the task flow for aproject. In another embodiment, the processing device may automaticallyupdate the task flow for the project, the task, and/or the firstnotification based on the user response. The update may aid subsequentusers performing the same task.

In another embodiment, the first notification and/or the secondnotification may be associated with user safety notifications. Asdiscussed above, the sensors of the wearable device 100 may takemeasurements related to user safety, such as temperature measurements,radiation measurements, humidity measurements, and so forth. Theprocessing device may analyze the measurements by the sensors todetermine whether the user's safety is in danger and display anotification while the user is performing the task or after the task iscomplete to provide messages for safety protocols and/or how to adjustthe users performance of the task to perform is more safely. Forexample, when a user is performing a task to repair a part on an oil rigand the user safety information indicates that the ambient temperatureexceeds a threshold temperature, the first notification or the secondnotification may notify the user to stop performing the task until theambient temperature decrease below the threshold temperature. In anotherexample, the optimizer 314 may send a notification to another device,such as a device associated with a supervisor or a manager, to indicatethe user safety dangers for the supervisor or manager to review. Thesupervisor or manager may adjust tasks to reduce the danger to the user.

In another embodiment, the first notification and/or the secondnotification may notify the user that the user is performing extrastep(s) or activities when performing a task by comparing the activitiesof the user when performing a task to the activities of other users whenperforming the task. For example, the processing device may access adatabase storing information indicating the activities performed byanother user while performing a task and may compare those activitieswith the activities performed by the user (such as those activitiesmeasured by sensors). When the user's activities are different than theother users and/or the user's activities include additional activitiesnot performed by the other users, the first notification and/or thesecond notification may indicate to the user the different and/oradditional activities performed by the user.

The method may include sending a third notification to another deviceindicating an improvement to the task or the task flow for a project(block 614). In one embodiment, the tasks of a task flow and thethreshold amounts of time may be generated by a task flow creator. Asusers are performing the tasks in the task flow for a project, theprocessing device may send the third notifications to the deviceassociated with the task flow creator. In one embodiment, thenotifications may include raw or unfiltered data and the task flowcreator may review the data to determine if the task flow creator canadjust the task flow to increase efficiencies, decrease safety issues,and/or decrease issues faced by the users when performing the tasks inthe task flow.

In another embodiment, the notification may include messages indicatingissues the users faced while performing the task(s) and/or suggestionsfor how the task flow may be adjusted to increase efficiencies, decreasesafety issues, and/or decrease issues faced by the users when performingthe tasks in the task flow. For example, the task flow creator mayreceive notifications indicating an amount of time the users are takingto perform each task and suggestions to adjust the task flow to increaseefficiencies, decrease safety issues, and/or decrease issues faced bythe users when performing the tasks in the task flow based on theactivities of other individuals previously performed the same task.

In one example, the task flow creator may be an automotive servicecenter technician optimizing an augmented reality experience for servicetask flows. A task flow may range from regular inspection to a complexrepair. For each task, data regarding the tasks of the task flow may becollected. The processing device may compare the data with a data modelto identify agreements and/or deviations by the user in performing thetasks. When the duration to complete and the amount of time to perform atask exceeds a predefined threshold, the processing device may acquireand/or analyze secondary data indicating the activities performed by theuser when performing the tasks, such as the user's hand gesture or headmotion to determine the type of task the user is performing, theactivities of the user and/or the tools the user is using to perform thetask. The processing device may then display the first notificationand/or the second notification to provide suggestion to the user onsupporting documents, videos, recordings related to the task and/orestablish a communication link with an expert to receive aid inperforming the task.

The method may include adjusting the at least one of the wearable device100 in FIGS. 1-5 , the task flow, a message associate with the task, thefirst notification, the second notification, or the third notificationbased on the collected data (block 616). For example, the processingdevice may adjust a display of the wearable device 100 in FIGS. 1-5based on the sensor data and/or the user exceeding the threshold amountof time to perform the tasks. Adjusting the display may include changingconfiguration of the display, such as a lighting level of the display toimprove visibility, altering color schemes of the display to improvecontrast, and so forth.

In another embodiment, a task flow owner may manually adjust the settingof the wearable device, the first notification, and/or the secondnotification based on the sensor data and/or the activities beingperformed by the user. In another embodiment, the processing device mayadjust the settings of the wearable device 100 to determine the settingsincrease a user's performance when performing tasks. For example, theprocessing device may perform A/B testing of different settings of thewearable device 100 and analyze the amount of time a user takes toperform the task to determine which setting(s) increases the user'sperformance when performing the tasks. In another embodiment, theprocessing device may compare the data from environmental sensors todetermine the environmental conditions to increase the user'sperformance when performing a task.

In another embodiment, the task flow owner or the processing device mayadjust the task flow by changing one or more of the tasks of the taskflow. For example, the task flow owner may add or remove a task in themap of the task flow. In another embodiment, the task flow owner or theprocessing device may adjust a message associated with the task. Forexample, originally the message may instruct the user regarding the nextstep of the task and the task flow owner or the processing device mayadjust or change the message to clarify what the task is and/or whatactivities the user is to perform to complete the task. In anotherembodiment, the task flow owner or the processing device may adjust thefirst notification to aid the user in completing the task on time. Forexample, the original first notification may have been unclear orimprecise regarding the suggestion or aid for the user to complete thetask and task flow owner or the processing device may adjust the firstnotification to clarify the suggestion or aid. In another example, thetask flow owner or the processing device may change the expert orexperienced individual with which the user in connected with to aid incompleting the task.

In another embodiment, the task flow owner or the processing device mayadjust the second notification to aid the user in understanding whatcaused the user to not complete the task on time. For example, theoriginal second notification may have been unclear or impreciseregarding the suggestion or aid for the user to complete the task nexttime the user performs the task and the task flow owner or theprocessing device may adjust the second notification to clarify thesuggestion or aid. In another embodiment, the processing device mayadjust the third notification to task flow owner in understanding whatcaused the user to not complete the task on time. For example, when theuser does not complete the task within the threshold amount of time fora second time, the processing device may include additional or differentsensor information and/or input from the user in the third notificationto aid the task flow owner in revising the task flow.

In another embodiment, when the wearable device, the task flow, themessage associate with the task, the first notification, the secondnotification, and/or the third notification have been adjusted themethod may end. In another embodiment, when the wearable device, thetask flow, the message associate with the task, the first notification,the second notification, and/or the third notification have beenadjusted the method may include iteratively selecting other tasks of thetask flow to adjust via the same method.

FIG. 6B illustrates a flowchart 620 for a method to adjust task in atask flow, according to an embodiment. In one embodiment, the method inthe flowchart 620 may be performed by the optimizer 314 in FIGS. 3 and 4executing on a processing device. As discussed above, the processingdevice may collect and analyze sensor data indicating activitiesperformed by a user while performing a task and determine task flowimprovements for the activities. These improvements may be shortenedtask flows or suggesting activities to eliminate. The improvements maybe performed while the user is performing the task and/or after the taskor project has been completed.

The method may include identifying a task in a task flow to collect dataassociated with a user performing the task (block 622). In oneembodiment, the identified tasks in the task flow may be identified astasks to optimize or tasks needing review. The identified tasks may bechosen for analysis by the task flow owner, the user, and/or theprocessing device.

The method may include collecting data for the identified task (block624). For example, the user may wear the wearable device 100 in FIGS.1A-5 and when the user starts a task flow for a project, a userinterface (UI) may display a first task of the task flow at a display ofthe wearable device 100. When the display displays the first task, theprocessing device of the wearable device 100 may initiate a first timerfor the task and collect data from one or more sensors of the wearabledevice 100. When the user completes the task and/or uses the UI toproceed to the next task in the task flow, the processing device maystop the first timer, tag the amount of time from the first timer withthe first task, and start a second timer for a second task of the taskflow. The processing device may initiate a new timer or reset thecurrent timer to measure the amount of time a user takes to complete atask and associate the amount of time with the task.

The method may include identifying a data model based on a user profilefor the user (block 626). In one example, the data and the amount oftime taken by the user may be compared to the data and amount of timetaken by an expert or an average of task performers. The processingdevice may determine a difference between the collected data and amountof time and the data model. For example, to determine a differencebetween the amount of time the user expended to perform a task and theamount of time an expert expended to perform the task, the processingdevice may subtract the amount of time taken by the user from the amountof time taken by the expert. In another example, to determine adifference between the sensor data of the user and the sensor data ofthe expert, the processing device may subtract the data of the user fromdata of the expert. In this example, when the data is movement data, theprocessing device may determine if the amount of movement by the userwas greater than the amount of movement by the expert by subtractingvalues representative of the amount of movement. The processing devicemay similarly determine the difference for other types of sensor datasuch as position data, speed data, angle data, and so forth.

The method may include comparing an amount of time the user expends toperform the task to a threshold amount of time associated with the datamodel (block 628). If the difference exceeds the threshold, theprocessing device may display a suggestion for how the user may increasehis/her performance when performing the task of the task flow. If thedifference is below the threshold, the processing device may display nosuggestion or may display a message encouraging or congratulating theuser. As discussed above, the thresholds may be based on the experiencelevel of the user, the industry associated with the task flow, and soforth. Also as discussed above, the sensors may collect environmentaldata, such as light parameters such as hue, intensity, and illuminationlevel for an environment approximate the user. The processing device maygenerate or adjust the threshold for the amount of time or the data inview of the environmental data. In another embodiment, the processingdevice may display via the display recommended environmentalcharacteristics for the user to perform the task, such as the lightinglevels, noise levels, weather conditions, and so forth.

The method may include providing a first notification to the user (suchas via the display) and/or providing a second notification to the ownerof the task flow via a device associated with the owner once the usercompletes the task or a set of tasks (block 630). For example, theprocessing device may send a device associated with the owner a messageto display to the owner alerting the owner regarding an amount of timeor other data for a single user or a set of user performing one or moretasks of the task flow. In another example, the processing device maydisplay the first notification to the user via the display in FIGS. 1A-5to indicate that the user is performing extra activities or steps whencompared to their peers or the next level of skill user. The firstnotification may also include suggestions to the user to communicatewith the owner or an expert for the task or the project.

The first notification may also include suggestions to the user on howpeers performed or resolved the task. For example, a database may storethe data for the user and/or other users performing the task and theprocessing device may query to the database to identifying users withthe same or similar sensor data and/or time data as the user, indicatingthe identified users may have encountered the same or similardifficulties for the task as the current user. The processing device maythen retrieve information in the database associated with the identifiedusers, where the information may indicate how the identified usersresolved the issues or problems they encountered while performing thetask or project. The processing device may then display the informationto the current user and/or suggestions based on the information.

In one example, the user may be an automotive service center technicianusing the wearable device 100 while serving vehicles. The projects forthe task flows may range from regular vehicle inspection to complexautomotive repairs. For each project, data associated with the user maybe collected by the sensors and processing device of the wearable device100. The data may include time duration data, location data, temperaturedata, pressure sensor data, and so forth. The processing device maycompare the data to a data model or model profile. As discussed above,when the data exceeds a threshold level, the processing device mayacquire secondary data, for the same sensors and/or other sensors. Forexample, the secondary data may indicate the tools used by a user toperform the task, head motion data indicating a type of task the user isperforming, and so forth. The processor suggests to the user supportingdocuments, videos, recordings related to the task. After a secondduration is exceeded, the processor offers to contact the expertsrelated to the task and arranges a conference call.

In another embodiment, the processing device may send a message to thedevice associated with the owner when the project is completedindicating improvements for the user to more efficiently perform thetask(s) of the task flow. In one example, the message may indicate thatmultiple users are encountering the same issue when performing a task orset of tasks and the users are taking more time to complete the task orset of tasks than the threshold amount of time. The multiple usersencountering the same issue may indicate the task may need to be splitup into multiple tasks, the task flow is missing a task, and/or thatadditional time may be needed to complete the task or set of tasks.

In another embodiment, the processing device may continuously monitorthe user while the user is performing a task or set of tasks for aproject. As discussed above, the data collected may relate to usersafety data, such as temperature data, radiation data, humidity data,safety data, and so forth. The processing device may analyze the usersafety data to determine when the user may be in danger (such as whenthe ambient temperature or noise level is too high) and the processordevice may display via the display a safety task flow to aid the user toresolve or avoid the dangerous situation.

FIG. 7 is a block diagram of a user device 700 with a processing devicein which implementations of methods, systems, devices, or apparatuses inFIGS. 1-6B may be implemented. The user device 700 may display and/orimplement the method and GUIs of FIGS. 1A-6B. The user device 700 may beany type of computing device such as an electronic book reader, a PDA, amobile phone, a laptop computer, a portable media player, a tabletcomputer, a camera, a video camera, a netbook, a desktop computer, agaming console, a DVD player, a computing pad, a media center, and thelike. The user device 700 may be any portable or stationary user device.For example, the user device 700 may be an intelligent voice control andspeaker system. Alternatively, the user device 700 can be any otherdevice used in a WLAN network (e.g., Wi-Fi® network), a WAN network, orthe like.

The user device 700 includes one or more processing device(s) 710, suchas one or more CPUs, microcontrollers, field programmable gate arrays,or other types of processing devices. The user device 700 also includessystem memory 702, which may correspond to any combination of volatileand/or non-volatile storage mechanisms. The system memory 702 storesinformation that provides operating system 704, various program modules706, program data 708, and/or other components. In one implementation,the system memory 702 stores instructions of the methods in FIGS. 6A and6B as described herein. The user device 700 performs functions by usingthe processing device(s) 710 to execute instructions provided by thesystem memory 702.

The user device 700 also includes a data storage device 716 that may becomposed of one or more types of removable storage and/or one or moretypes of non-removable storage. The data storage device 716 includes acomputer-readable storage medium 718 on which is stored one or more setsof instructions embodying any of the methodologies or functionsdescribed herein. Instructions for the program modules 706 may reside,completely or at least partially, within the computer-readable storagemedium 718, system memory 702 and/or within the processing device(s) 710during execution thereof by the user device 700, the system memory 702and the processing device(s) 710 also constituting computer-readablemedia. The user device 700 may also include one or more input devices712 (keyboard, mouse device, specialized selection keys, etc.) and oneor more output devices 714 (displays, printers, audio output mechanisms,etc.).

The user device 700 further includes modem 720 to allow the user device700 to communicate via a wireless network(s) (e.g., such as provided bythe wireless communication system) with other computing devices, such asremote computers, an item providing system, and so forth. The modem 720can be connected to zero or more RF modules 722. The zero or more RFmodules 722 can be connected to zero or more coupler circuitry 724. TheRF modules 722 and/or the coupler circuitry 724 may be a WLAN module, aWAN module, PAN module, or the like. Antenna 726 is coupled to thecoupler circuitry 724, which is coupled to the modem 720 via the RFmodules 722. The modem 720 allows the user device 700 to handle bothvoice and non-voice communications (such as communications for textmessages, multimedia messages, media downloads, web browsing, etc.) witha wireless communication system. The modem 720 may provide networkconnectivity using any type of mobile network technology including, forexample, cellular digital packet data (CDPD), general packet radioservice (GPRS), EDGE, universal mobile telecommunications system (UMTS),1 times radio transmission technology (1×RTT), evaluation data optimized(EVDO), high-speed downlink packet access (HSDPA), Wi-Fi® technology,Long Term Evolution (LTE) and LTE Advanced (sometimes generally referredto as 4G), etc.

The modem 720 may generate signals and send these signals to antenna 726via coupler circuitry 724 as described herein. User device 700 mayadditionally include a WLAN module, a GPS receiver, a PAN transceiverand/or other RF modules. The coupler circuitry 724 may additionally oralternatively be connected to one or more of coupler arrays. The antenna726 may be configured to transmit in different frequency bands and/orusing different wireless communication protocols. The antenna 726 may bedirectional, omnidirectional, or non-directional antennas. In additionto sending data, the antenna 726 may also receive data, which is sent toappropriate RF modules 722 connected to the antenna 726.

In one implementation, the user device 700 establishes a firstconnection using a first wireless communication protocol, and a secondconnection using a different wireless communication protocol. The firstwireless connection and second wireless connection may be activeconcurrently, for example, if a user device is downloading a media itemfrom a server (e.g., via the first connection) and transferring a fileto another user device (e.g., via the second connection) at the sametime. Alternatively, the two connections may be active concurrentlyduring a handoff between wireless connections to maintain an activesession (e.g., for a telephone conversation). Such a handoff may beperformed, for example, between a connection to a WLAN hotspot and aconnection to a wireless carrier system. In one implementation, thefirst wireless connection is associated with a first resonant mode of anantenna structure that operates at a first frequency band and the secondwireless connection is associated with a second resonant mode of theantenna structure that operates at a second frequency band. In anotherimplementation, the first wireless connection is associated with a firstantenna element and the second wireless connection is associated with asecond antenna element. In other implementations, the first wirelessconnection may be associated with a media purchase application (e.g.,for downloading electronic books), while the second wireless connectionmay be associated with a wireless ad hoc network application. Otherapplications that may be associated with one of the wireless connectionsinclude, for example, a game, a telephony application, an Internetbrowsing application, a file transfer application, a global positioningsystem (GPS) application, and so forth.

Though modem 720 is shown to control transmission and reception via theantenna 726, the user device 700 may alternatively include multiplemodems, each of which is configured to transmit/receive data via adifferent antenna and/or wireless transmission protocol.

The user device 700 delivers and/or receives items, upgrades, and/orother information via the network. For example, the user device 700 maydownload or receive items from an item providing system. The itemproviding system receives various requests, instructions and other datafrom the user device 700 via the network. The item providing system mayinclude one or more machines (e.g., one or more server computer systems,routers, gateways, etc.) that have processing and storage capabilitiesto provide the above functionality. Communication between the itemproviding system and the user device 700 may be enabled via anycommunication infrastructure. One example of such an infrastructureincludes a combination of a wide area network (WAN) and wirelessinfrastructure, which allows a user to use the user device 700 topurchase items and consume items without being tethered to the itemproviding system via hardwired links. The wireless infrastructure may beprovided by one or multiple wireless communications systems, such as oneor more wireless communications systems. One of the wirelesscommunication systems may be a wireless local area network (WLAN)hotspot connected to the network. The WLAN hotspots can be created byproducts based on IEEE 802.11x standards for the Wi-Fi® technology byWi-Fi® Alliance. Another of the wireless communication systems may be awireless carrier system that can be implemented using various dataprocessing equipment, communication towers, etc. Alternatively, or inaddition, the wireless carrier system may rely on satellite technologyto exchange information with the user device 700.

The communication infrastructure may also include acommunication-enabling system that serves as an intermediary in passinginformation between the item providing system and the wirelesscommunication system. The communication-enabling system may communicatewith the wireless communication system (e.g., a wireless carrier) via adedicated channel, and may communicate with the item providing systemvia a non-dedicated communication mechanism, e.g., a public Wide AreaNetwork (WAN) such as the Internet.

The user device 700 is variously configured with different functionalityto enable consumption of one or more types of media items. The mediaitems may be any type of format of digital content, including, forexample, electronic texts (e.g., eBooks, electronic magazines, digitalnewspapers, etc.), digital audio (e.g., music, audible books, etc.),digital video (e.g., movies, television, short clips, etc.), images(e.g., art, photographs, etc.), and multi-media content. The userdevices 700 may include any type of content rendering devices such aselectronic book readers, portable digital assistants, mobile phones,laptop computers, portable media players, tablet computers, cameras,video cameras, netbooks, notebooks, desktop computers, gaming consoles,DVD players, media centers, and the like.

In the above description, numerous details are set forth. It will beapparent, however, to one of ordinary skill in the art having thebenefit of this disclosure, that implementations may be practicedwithout these specific details. In some instances, well-known structuresand devices are shown in block diagram form, rather than in detail, inorder to avoid obscuring the description.

Some portions of the detailed description are presented in terms ofalgorithms and symbolic representations of operations on data bitswithin a computer memory. These algorithmic descriptions andrepresentations are the means used by those skilled in the dataprocessing arts to most effectively convey the substance of their workto others skilled in the art. An algorithm is here, and generally,conceived to be a self-consistent sequence of steps leading to thedesired result. The steps are those requiring physical manipulations ofphysical quantities. Usually, though not necessarily, these quantitiestake the form of electrical or magnetic signals capable of being stored,transferred, combined, compared, and otherwise manipulated. It hasproven convenient at times, principally for reasons of common usage, torefer to these signals as bits, values, elements, symbols, characters,terms, numbers or the like.

It should be borne in mind, however, that all of these and similar termsare to be associated with the appropriate physical quantities and aremerely convenient labels applied to these quantities. Unlessspecifically stated otherwise as apparent from the above discussion, itis appreciated that throughout the description, discussions utilizingterms such as “inducing,” “parasitically inducing,” “radiating,”“detecting,” determining,” “generating,” “communicating,” “receiving,”“disabling,” or the like, refer to the actions and processes of acomputer system, or similar electronic computing device, thatmanipulates and transforms data represented as physical (e.g.,electronic) quantities within the computer system's registers andmemories into other data similarly represented as physical quantitieswithin the computer system memories or registers or other suchinformation storage, transmission or display devices.

Implementations also relate to an apparatus for performing theoperations herein. This apparatus may be specially constructed for therequired purposes, or it may comprise a general-purpose computerselectively activated or reconfigured by a computer program stored inthe computer. Such a computer program may be stored in acomputer-readable storage medium, such as, but not limited to, any typeof disk including floppy disks, optical disks, CD-ROMs andmagnetic-optical disks, read-only memories (ROMs), random accessmemories (RAMs), EPROMs, EEPROMs, magnetic or optical cards, or any typeof media suitable for storing electronic instructions.

The algorithms and displays presented herein are not inherently relatedto any particular computer or other apparatus. Various general-purposesystems may be used with programs in accordance with the teachingsherein, or it may prove convenient to construct a more specializedapparatus to perform the required method steps. The required structurefor a variety of these systems will appear from the description below.In addition, the present implementations are not described withreference to any particular programming language. It will be appreciatedthat a variety of programming languages may be used to implement theteachings of the present invention as described herein. It should alsobe noted that the terms “when” or the phrase “in response to,” as usedherein, should be understood to indicate that there may be interveningtime, intervening events, or both before the identified operation isperformed.

It is to be understood that the above description is intended to beillustrative, and not restrictive. Many other implementations will beapparent to those of skill in the art upon reading and understanding theabove description. The scope of the present implementations should,therefore, be determined with reference to the appended claims, alongwith the full scope of equivalents to which such claims are entitled.

In the above description, numerous details are set forth. It will beapparent, however, to one of ordinary skill in the art having thebenefit of this disclosure, that implementations may be practicedwithout these specific details. In some instances, well-known structuresand devices are shown in block diagram form, rather than in detail, inorder to avoid obscuring the description.

Although the implementations may be herein described with reference tospecific integrated circuits, such as in computing platforms ormicroprocessing devices, other implementations are applicable to othertypes of integrated circuits and logic devices. Similar techniques andteachings of implementations described herein may be applied to othertypes of circuits or semiconductor devices. For example, the disclosedimplementations are not limited to desktop computer systems orUltrabooks™ and may be also used in other devices, such as handhelddevices, tablets, other thin notebooks, systems on a chip (SOC) devices,and embedded applications. Some examples of handheld devices includecellular phones, Internet protocol devices, smartphones, digitalcameras, personal digital assistants (PDAs), and handheld PCs. Embeddedapplications typically include a microcontroller, a digital signalprocessing device (DSP), a system on a chip, network computers (NetPC),set-top boxes, network hubs, wide area network (WAN) switches, or anyother system that may perform the functions and operations taught below.

Although the implementations are herein described with reference to aprocessing device or processing device, other implementations areapplicable to other types of integrated circuits and logic devices.Similar techniques and teachings of implementations of the presentinvention may be applied to other types of circuits or semiconductordevices that may benefit from higher pipeline throughput and improvedperformance. The teachings of implementations of the present inventionare applicable to any processing device or machine that performs datamanipulations. However, the present invention is not limited toprocessing devices or machines that perform 512 bit, 256 bit, 128 bit,64 bit, 32 bit, and/or 16 bit data operations and may be applied to anyprocessing device and machine in which manipulation or management ofdata is performed. In addition, the following description providesexamples, and the accompanying drawings show various examples for thepurposes of illustration. However, these examples should not beconstrued in a limiting sense as they are merely intended to provideexamples of implementations of the present invention rather than toprovide an exhaustive list of all possible implementations ofimplementations of the present invention.

Some portions of the detailed description are presented in terms ofalgorithms and symbolic representations of operations on data bitswithin a computer memory. These algorithmic descriptions andrepresentations are the means used by those skilled in the dataprocessing arts to most effectively convey the substance of their workto others skilled in the art. An algorithm is here and generally,conceived to be a self-consistent sequence of operations leading to adesired result. The operations are those requiring physicalmanipulations of physical quantities. Usually, though not necessarily,these quantities take the form of electrical or magnetic signals capableof being stored, transferred, combined, compared and otherwisemanipulated. It has proven convenient at times, principally for reasonsof common usage, to refer to these signals as bits, values, elements,symbols, characters, terms, numbers or the like. The blocks describedherein may be hardware, software, firmware, or a combination thereof.

It should be borne in mind, however, that all of these and similar termsare to be associated with the appropriate physical quantities and aremerely convenient labels applied to these quantities. Unlessspecifically stated otherwise as apparent from the above discussion, itis appreciated that throughout the description, discussions utilizingterms such as “detecting,” “initiating,” “determining,” “continuing,”“halting,” “receiving,” “recording,” or the like, refer to the actionsand processes of a computing system, or similar electronic computingdevice, that manipulates and transforms data represented as physical(e.g., electronic) quantities within the computing system's registersand memories into other data similarly represented as physicalquantities within the computing system memories or registers or othersuch information storage, transmission or media devices.

The words “example” or “exemplary” are used herein to mean serving as anexample, instance or illustration. Any aspect or design described hereinas “example’ or “exemplary” is not necessarily to be construed aspreferred or advantageous over other aspects or designs. Rather, use ofthe words “example” or “exemplary” is intended to present concepts in aconcrete fashion. As used in this application, the term “or” is intendedto mean an inclusive “or” rather than an exclusive “or.” That is, unlessspecified otherwise, or clear from context, “X includes A or B” isintended to mean any of the natural inclusive permutations. That is, ifX includes A; X includes B; or X includes both A and B, then “X includesA or B” is satisfied under any of the foregoing instances. In addition,the articles “a” and “an” as used in this application and the appendedclaims should generally be construed to mean “one or more” unlessspecified otherwise or clear from context to be directed to a singularform. Moreover, use of the term “an implementation” or “oneimplementation” or “an implementation” or “one implementation”throughout is not intended to mean the same implementation orimplementation unless described as such. Also, the terms “first,”“second,” “third,” “fourth,” etc. as used herein are meant as labels todistinguish among different elements and may not necessarily have anordinal meaning according to their numerical designation.

Implementations described herein may also relate to an apparatus forperforming the operations herein. This apparatus may be speciallyconstructed for the required purposes, or it may comprise a computerselectively activated or reconfigured by a computer program stored inthe computer. Such a computer program may be stored in a non-transitorycomputer-readable storage medium, such as, but not limited to, any typeof disk including floppy disks, optical disks, CD-ROMs andmagnetic-optical disks, read-only memories (ROMs), random accessmemories (RAMs), EPROMs, EEPROMs, magnetic or optical cards, flashmemory, or any type of media suitable for storing electronicinstructions. The term “computer-readable storage medium” should betaken to include a single medium or multiple media (e.g., a centralizedor distributed database and/or associated caches and servers) that storethe one or more sets of instructions. The term “computer-readablemedium” shall also be taken to include any medium that is capable ofstoring, encoding or carrying a set of instructions for execution by themachine and that causes the machine to perform any one or more of themethodologies of the present implementations. The term“computer-readable storage medium” shall accordingly be taken toinclude, but not be limited to, solid-state memories, optical media,magnetic media, any medium that is capable of storing a set ofinstructions for execution by the machine and that causes the machine toperform any one or more of the methodologies of the presentimplementations.

The algorithms and displays presented herein are not inherently relatedto any particular computer or other apparatus. Various computing systemsmay be used with programs in accordance with the teachings herein, or itmay prove convenient to construct a more specialized apparatus toperform the operations. The required structure for a variety of thesesystems will appear from the description below. In addition, the presentimplementations are not described with reference to any particularprogramming language. It will be appreciated that a variety ofprogramming languages may be used to implement the teachings of theimplementations as described herein.

The above description sets forth numerous specific details such asexamples of specific systems, components, methods and so forth, in orderto provide a good understanding of several implementations. It will beapparent to one skilled in the art, however, that at least someimplementations may be practiced without these specific details. Inother instances, well-known components or methods are not described indetail or are presented in simple block diagram format in order to avoidunnecessarily obscuring the present implementations. Thus, the specificdetails set forth above are merely exemplary. Particular implementationsmay vary from these exemplary details and still be contemplated to bewithin the scope of the present implementations.

It is to be understood that the above description is intended to beillustrative and not restrictive. Many other implementations will beapparent to those of skill in the art upon reading and understanding theabove description. The scope of the present implementations should,therefore, be determined with reference to the appended claims, alongwith the full scope of equivalents to which such claims are entitled.

The disclosure above encompasses multiple distinct embodiments withindependent utility. While these embodiments have been disclosed in aparticular form, the specific embodiments disclosed and illustratedabove are not to be considered in a limiting sense as numerousvariations are possible. The subject matter of the embodiments includesthe novel and non-obvious combinations and sub-combinations of thevarious elements, features, functions and/or properties disclosed aboveand inherent to those skilled in the art pertaining to such embodiments.Where the disclosure or subsequently filed claims recite “a” element, “afirst” element, or any such equivalent term, the disclosure or claims isto be understood to incorporate one or more such elements, neitherrequiring nor excluding two or more such elements.

Applicant(s) reserves the right to submit claims directed tocombinations and sub-combinations of the disclosed embodiments that arebelieved to be novel and non-obvious. Embodiments embodied in othercombinations and sub-combinations of features, functions, elementsand/or properties may be claimed through amendment of those claims orpresentation of new claims in the present application or in a relatedapplication. Such amended or new claims, whether they are directed tothe same embodiment or a different embodiment and whether they aredifferent, broader, narrower or equal in scope to the original claims,are to be considered within the subject matter of the embodimentsdescribed herein.

The invention claimed is:
 1. A device, comprising: a head-mounteddisplay configured to display a virtual object in an augmented realityenvironment; a sensor configured to monitor a user performing a task; aprocessing device coupled to the head-mounted display and the sensor,wherein the processing device is configured to: identify the task in atask flow for a project to collect data associated with the userperforming the task; display a message associated with the task; collectdata for the task using the sensor; identify a data model based on auser profile for the user; compare an amount of time the user expends toperform the task to a threshold amount of time associated with the datamodel; and in response to the amount of time the user expends to performthe task exceeding the threshold amount of time: provide a firstnotification to the user while performing the task; provide a secondnotification to the user when the task or the project is completed; senda third notification to another device indicating an improvement to thetask or the task flow for the project; and adjust at least one of thedevice, the task flow, the message associate with the task, the firstnotification, the second notification, or the third notification basedon the collected data.
 2. The device of claim 1, wherein the firstnotification includes a suggestion to aid the user in performing thetask.
 3. The device of claim 1, wherein the first notification includescommunication link to another device associated with an expert or anindividual with experience performing the task.
 4. The device of claim1, wherein the first notification includes a request for the user toprovide feedback via an input device indicating whether the firstnotification was helpful to the user in completing the task.
 5. Thedevice of claim 1, wherein the second notification includes acommunication link to another device associated with an expert or anindividual with experience performing the task.
 6. The device of claim1, wherein the second notification includes a message indicating theuser performed an extra step or an extra activity when performing thetask.
 7. The device of claim 1, wherein the first notification or thesecond notification include a safety notification.
 8. The device ofclaim 1, wherein the third notification includes a message indicating animprovement to the task or the task flow based on the collected data. 9.The device of claim 1, wherein the processing device is configured to,in response to the amount of time the user expends to perform the tasknot exceeding the threshold amount of time, select another task of thetask flow to compare to another threshold amount of time and provide anassociated notification for.
 10. The device of claim 1, wherein toadjust the device the processing device is configured to adjust aconfiguration of the head-mounted display based on the collected data.11. The device of claim 10, wherein to adjust the configuration of thehead-mounted display the processing device is configure to adjust alighting configuration of the head-mounted display.
 12. The device ofclaim 1, wherein the processing device is configured to adjust the taskflow based on the collected data.
 13. The device of claim 1, wherein theprocessing device is configured to adjust the message associated withthe task, the first notification, the second notification, or the thirdnotification based on the collected data.
 14. A device, comprising: adisplay configured to display a virtual object in an augmented realityenvironment or a virtual reality environment; a sensor configured tomonitor a user performing a task; and a processing device coupled to thedisplay and the sensor, wherein the processing device is configured to:identify the task in a task flow for a project to collect dataassociated with the user performing the task; collect data for the taskusing the sensor; compare an amount of time the user expends to performthe task to a threshold amount of time associated with a user profilefor the user; and in response to the amount of time the user expends toperform the task exceeding the threshold amount of time: display a firstnotification to the user while performing the task; display a secondnotification to the user when the task or the project is completed; orsend a third notification to another device indicating an improvement tothe first task or the task flow.
 15. The device of claim 14, wherein theprocessing device is further configured to, in response to the amount oftime the user expends to perform the task exceeding the threshold amountof time, adjust at least one of the device, the task flow, the firstnotification, or the second notification based on the collected data.16. A method, comprising: identifying, by a processing device, a firsttask in a task flow to collect data associated with a user performingthe first task; collecting, by a sensor, a first set of data associatedwith the user performing the first task; comparing, by the processingdevice, an amount of time the user expends to perform the first task toa first threshold amount of time; and in response to the amount of timethe user expends to perform the first task exceeding the first thresholdamount of time: displaying, by the display, a first notification to theuser while performing the first task; displaying, by the display, asecond notification to the user when the first task or the project iscompleted; sending, by the processing device, a third notification toanother device indicating an improvement to the first task or the taskflow for a project; or adjusting, by the processing device, the at leastone of the task flow, the first notification, the second notification,or the third notification based on the first set of data.
 17. The methodof claim 16, wherein the first threshold amount of time corresponds to auser profile associated with the user.
 18. The method of claim 17,wherein the first threshold amount of time is based on an experiencefeature of the user, an industry associated with the first task, or anenvironmental condition in an area where the first task is performed bythe user.
 19. The method of claim 16, further comprising: identifying,by the processing device, a second task in the task flow to collect dataassociated with the user performing the second task; collecting, by thesensor, a second set of data associated with the user performing thesecond task; comparing, by the processing device, an amount of time theuser expends to perform the second task to a second threshold amount oftime; and in response to the amount of time the user expends to performthe second task exceeding the second threshold amount of time:displaying, by the display, a fourth notification to the user whileperforming the second task; displaying, by the display, a fifthnotification to the user when the second task or the project iscompleted; sending, by the processing device, a sixth notification tothe other device indicating an improvement to the second task or thetask flow for the project; or adjusting, by the processing device, atleast one of the task flow, the fourth notification, the fifthnotification, or the sixth notification based on the second set of data.